A Neural Network Approach to Interactive Content-Based Retrieval of Video Databases

A neural network scheme is presented in this paper for adaptive video indexing and retrieval. First, a limited but characteristic amount of frames are extracted from each video scene, by minimizing a cross-correlation criterion. Low level features are extracted to indicate the frame characteristics, such as color and motion segments. This is due to the fact that extraction of high-level, semantic, features from any kind of images is too hard to be implemented. After the key frame extraction, the video queries are implemented directly on this small number of frames. To reduce, however, the limitation of low-level features, the human is considered as a part of the process, meaning that he/she is able to assign a degree of appropriateness for each retrieved image of the system and then restart the searching. A feedforward neural network structure is proposed as a parametric distance for the retrieval, mainly due to the highly non linear capabilities. An adaptation mechanism is also proposed for updating the network weights, each time a new image selection is performed by the user. This mechanism can modify the network weights so that the output of the network, after the adaptation, is as much as close to the user’s selection while simultaneously performing a minimal degradation of the previous learned data.